The present invention generally relates to data processing. The invention relates more specifically to computer software and systems useful in controlling computer graphic characters and electromechanically actuated puppet characters as used in the production of motion pictures.
In the performing arts, puppetry traditionally has been a manually performed art that relies on the skill, creativity, and coordinated movement of a human puppeteer who manipulates a passive, mechanical puppet. Sophisticated artificial characters or xe2x80x9ccreaturesxe2x80x9d that resemble puppets, developed by Jim Henson and The Jim Henson Company, offer a wide range of motion and character expression. However, generally they require a human performer to hold them or otherwise use close manual control. This is a disadvantage in certain types of motion picture productions that feature large or fully autonomous creatures that cannot be held by or otherwise directly manipulated by a human performer.
Partly in response to this problem, remotely controlled, robotic electromechanically actuated puppet creatures have been developed. An electromechanically actuated puppet may simulate any desired animal, human character, or other creature, real or imagined, of any size. Typically an electromechanically actuated puppet comprises: a movable skeleton made of metals or other rigid materials; servos, motors, hydraulics or other actuators that move parts of the skeleton (limbs, facial features, eyes, etc.); and a skin that covers the skeleton and simulates real skin, muscle, hair, etc. Electromechanically actuated puppets also offer the ability to control complex expressions involving numerous actuation points in a small area Electromechanically actuated puppets may be remotely controlled using a cable tether that connects a remote onboard computer to a control computer, or using a wireless link. Electromechanically actuated puppets have been featured in such motion pictures and television productions as xe2x80x9cBabe,xe2x80x9d xe2x80x9cGeorge Of The Jungle,xe2x80x9d xe2x80x9cDr. Doolittle,xe2x80x9d xe2x80x9cAnimal Farm,xe2x80x9d and others.
Unfortunately, electromechanically actuated puppets are relatively costly, and require considerable time and skill to create, test, and debug. In some cases, the amount of time that a performer can spend in rehearsal time with the electromechanically actuated puppet is limited because of concerns about wearing out its mechanical parts. As a result, in past productions the performers of the electromechanically actuated puppets sometimes have had difficulty gaining familiarity with a new electromechanically actuated puppet. This is undesirable. It is desirable to have a way for the performer to have ample rehearsal time to create and test out expressions with less wear on the puppet.
Moreover, improvements in computer graphics technology and networking technology have lead to increased interest in creating and distributing performances of computer graphic (CG) characters. Unlike an electromechanically actuated puppet, a CG character is intangible and takes form only as an animated graphical display generated by a computer. CG characters may be created, for example, using 3D graphics computer software such as 3D Studio Max, commercially available from Kinetix, a unit of Autodesk, Inc. There is tremendous interest now in creating and performing CG characters for use in World Wide Web and video game content and productions. Conventional graphics software systems, however, require the performer to learn how to use the software in order to create expressions and performances for the CG characters. It is desirable to have a way for a performer to create expressions and performances for CG characters in a manner that is homologous to a puppetry performance, without having to use graphics software or other traditional computer input methods.
In particular, existing CG tools provide only low-level approaches for generating expressions and performances of characters. Typically they are complex, time consuming to use, and prone to user error. There is a need for a performance control system that offers users high-level configuration tools for rapid generation of expressions and performances.
Control systems used to manipulate electromechanically actuated puppets are known. For example, since approximately 1989, Jim Henson""s Creature Shop of London, England, a unit of The Jim Henson Company, has used a computer system known as xe2x80x9cBig1xe2x80x9d to control performances of electromechanically actuated puppet characters, as described in the document, xe2x80x9cBig1: User Manual,xe2x80x9d printed Aug. 23, 1990. Generally, the Big1 system comprises a dedicated, custom designed computer; a handset for providing manual input to the computer; a motor driver for receiving signals from the computer and driving motors of an electromechanically actuated puppet; a power supply; and an i/o unit known as Smarter Tool. The Big1 can provide real time control of 32 servos and 8 motor actuators by one puppeteer. The Jim Henson Company won an Academy Award for Technical Achievement for the design of this control system. Most recently the Big1 system was used in the production of xe2x80x9cJack Frostxe2x80x9d and xe2x80x9cFarscape.xe2x80x9d
In 1990 an edit suite called Performance Editor was developed which works in tandem with the Big1 control system to capture a performance of a puppeteer and edit a performance, to enable synchronization to a sound track for accurate lip-synching. The edit suite was first used in the production of the film xe2x80x9cBabe,xe2x80x9d for which The Jim Henson Company won an Academy Award for Best Special Effects.
The Big1 system, however, has certain known disadvantages. For example, use of a custom, dedicated control computer causes problems in modifying the system to add new features and functions. Such a computer cannot be expanded in terms of speed, memory, or peripherals in an easy or cost-effective manner. All principal functionality executes on the control computer, resulting a monolithic architecture that provides a single point of failure. A monolithic software architecture, without an operating system, is also used, making distribution of program components impractical. This architecture has no memory protection, no multi-tasking, no device support, and no application programming interface. Hence, the system lacks modularity, and therefore is difficult to update, modify, or extend. It also cannot use widely available applications and tools that are available for other operating systems.
Another fundamental disadvantage of the Big1 system is that it is not readily usable with computer graphic (CG) characters.
Based on the foregoing, there is a need for a way for the performer to create and test out expressions more easily and rapidly.
There is also a need for a way for a performer to create expressions and performances for CG characters in a manner that is homologous to a puppetry performance, without having to use graphics software or other traditional computer input methods.
These needs, and other needs that will become apparent from the following description, are fulfilled by the present invention, which comprises, in one aspect, a method and apparatus for performing a computer graphic character live in a manner homologous to a live puppetry performance. Character representation information is created and stored using a first computer. Performer movement information is received at the first computer from a manual input device that receives live manual manipulations and converts the manipulations into the performer movement information. Character motion information is created and stored based on combining the performer movement information with the character representation information. The character motion information is communicated in real time to a second computer. The second computer converts the character motion information into movements of a computer graphic character, which is displayed substantially synchronized to the live manual manipulations. Control objects define elements of the manual input device. Actuators define movable elements of the computer graphic character. Expression objects tie controls to actuators and may be used to develop and store complex character expressions. The system may also drive robotic or electromechanically actuated puppet creatures.
In another aspect, the invention provides a method of performing a computer graphic character live. The method involves creating and storing character representation information in a first computer; receiving, at the first computer, performer movement information from a manual input device that receives live manual manipulations and converts the manipulations into the performer movement information; creating and storing character motion information based on combining the performer movement information with the character representation information; receiving the character motion information at a second computer; displaying the computer graphic character substantially synchronized to the live manual manipulations using the second computer and based on the character motion information.
In another aspect, the invention provides a computer system providing live performance control of one or more computer graphic characters, comprising a first computer comprising a character document that defines attributes of one or more characters and an expression generator that creates and stores character motion information based on attributes in the character documents; a transducer apparatus coupled to the first computer to provide digital information representing movement to the first computer as the transducer apparatus is moved; a second computer coupled to the first computer and comprising a character generating element configured to generate display information depicting the one or more computer graphic characters in response to and based on the digital information and the character motion information; and a graphic display device coupled to the second computer and configured to display the computer graphic characters using the display information.
In still another aspect, the invention provides a computer system providing live performance control of computer graphic characters, comprising a computer that stores one or more character documents that define attributes of the characters and comprising an expression generator that creates and stores character motion information based on attributes in the character documents; a transducer apparatus coupled to the computer to provide digital information representing movement to the computer as the transducer apparatus is moved; a character generating element in the computer that is configured to generate display information depicting the computer graphic characters in response to and based on the digital information and the character motion information received via a local communications link; and a graphic display device coupled to the computer and configured to display the computer graphic characters using the display information and using a character generation application and a 3D display application.
In yet another aspect, the invention provides a computer system providing live performance control of computer graphic characters, comprising a computer that stores a character document that defines attributes of one or more characters and comprising an expression generator that creates and stores character motion information based on attributes in the character document; a character generating element in the computer that is configured to generate display information depicting the computer graphic characters in response to and based on the character motion information and digital information that is received from a transducer apparatus coupled to the computer, wherein the digital information represents movement to the computer as the transducer apparatus is moved; and a graphic display device coupled to the computer and configured to display the computer graphic characters using the display information.
In yet another aspect, the invention encompasses a computer-readable medium carrying one or more sequences of instructions for performing a computer graphic character live, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the steps of creating and storing character representation information in a first computer; receiving, at the first computer, performer movement information from a manual input device that receives live manual manipulations; creating and storing character motion information based on combining the performer movement information with the character representation information; receiving the character motion information at a second computer; displaying the computer graphic character substantially synchronized to the live manual manipulations using the second computer and based on the character motion information.
In still another aspect, the invention provides a method of producing a motion picture that includes a performance of a computer graphic character, comprising the steps of creating and storing character representation information in a first computer; receiving, at the first computer, performer movement information from a manual input device that receives live manual manipulations and converts the manipulations into the performer movement information; creating and storing character motion information based on combining the performer movement information with the character representation information; receiving the character motion information at a second computer; creating and storing a motion picture medium that includes a display of the computer graphic character substantially synchronized to the live manual manipulations using the second computer and based on the character motion information.
In yet another aspect, the invention provides a method of creating and storing information that defines a computer graphic character, comprising the steps of creating and storing character representation information in a first computer; receiving, at the first computer, performer movement information from a manual input device that receives live manual manipulations and converts the manipulations into the performer movement information; creating and storing character motion information based on combining the performer movement information with the character representation information; receiving the character motion information at a second computer; creating and storing a character model based on a display of the computer graphic character that is substantially synchronized to the live manual manipulations using the second computer and based on the character motion information.
Other features and aspects will become apparent from the following description.